Vital Sign Monitoring in Dynamic Environment via mmWave Radar and Camera Fusion

Contact-free vital sign monitoring, which uses wireless signals for recognizing human vital signs (i.e, breath and heartbeat), is an attractive solution to health and security. However, the subject's body movement and the change in actual environments can result in inaccurate frequency estimation of heartbeat and respiratory. In this paper, we propose a robust mmWave radar and camera fusion system for monitoring vital signs, which can perform consistently well in dynamic scenarios, e.g., when some people move around the subject to be tracked, or a subject waves his/her arms and marches on the spot. Three major processing modules are developed in the system, to enable robust sensing. Firstly, we utilize a camera to assist a mmWave radar to accurately localize the subjects of interest. Secondly, we exploit the calculated subject position to form transmitting and receiving beamformers, which can improve the reflected power from the targets and weaken the impact of dynamic interference. Thirdly, we propose a weighted multi-channel Variational Mode Decomposition (WMC-VMD) algorithm to separate the weak vital sign signals from the dynamic ones due to subject's body movement. Experimental results show that, the 90${^{th}}$ percentile errors in respiration rate (RR) and heartbeat rate (HR) are less than 0.5 RPM (respirations per minute) and 6 BPM (beats per minute), respectively

Modeling of Mass Balance Variability and Its Impact on Water Discharge from the Urumqi Glacier No. 1 Catchment, Tian Shan, China

Originating in the Tian Shan mountains, Urumqi River plays a key role in terms of water supply to downstream areas. In its headwaters, Urumqi Glacier No. 1 (UG1) is the largest glacier contributing to water discharge. Assessing its response to the changing climatic conditions in the area is of major importance to quantify future water availability. We here apply COSIPY, a COupled Snowpack and Ice surface energy and mass balance model in PYthon, to UG1, implementing a new albedo parameterization which integrates site-specific bare-ice albedo values on a pixel-by-pixel basis observed by remote sensing. We assess model performance threefold: quantitatively based on long-term measurement data of (1) surface mass balance (SMB) and (2) water discharge as well as qualitatively (3) comparing simulated snow line altitudes to such imated on the basis of time-lapse photography. Comparison of the modeled SMB with annually-averaged data from ablation stakes reveals that COSIPY including the new albedo parameterization accounts for 57.6% of the variance observed in the measurements. The original albedo parameterization performs only slightly inferior (57.1%). Glacier-wide comparison between modeled and glaciological SMB shows high agreement. In terms of discharge prediction, COSIPY reproduces onset and duration of the discharge season well. Estimated discharge from the whole catchment shows shortcomings in exactly matching the measured times series, but interannual variability is captured.Peer Reviewe

Local Global Relational Network for Facial Action Units Recognition

Many existing facial action units (AUs) recognition approaches often enhance the AU representation by combining local features from multiple independent branches, each corresponding to a different AU. However, such multi-branch combination-based methods usually neglect potential mutual assistance and exclusion relationship between AU branches or simply employ a pre-defined and fixed knowledge-graph as a prior. In addition, extracting features from pre-defined AU regions of regular shapes limits the representation ability. In this paper, we propose a novel Local Global Relational Network (LGRNet) for facial AU recognition. LGRNet mainly consists of two novel structures, i.e., a skip-BiLSTM module which models the latent mutual assistance and exclusion relationship among local AU features from multiple branches to enhance the feature robustness, and a feature fusion&refining module which explores the complementarity between local AUs and the whole face in order to refine the local AU features to improve the discriminability. Experiments on the BP4D and DISFA AU datasets show that the proposed approach outperforms the state-of-the-art methods by a large margin

Computer Vision-assisted Battery-free RFID Systems for Object Recognition, Localization and Orientation

University of Technology Sydney. Faculty of Engineering and Information Technology.Battery-free radio frequency identification (RFID) is a promising technique in Internet of Things (IoT) applications that use wireless signals to identify a physical object from its attached RFID tag. Compared to the existing barcode identification systems, RFID can still work in the non-line-of-sight (NLOS) scenarios that some obstructions block the identifier. Recently, many researchers start regarding each RFID tag as a battery-free sensor, whose indicator is the backscatter signal fingerprint reported by an RFID reader. Since the sensor could sense the change in the position and orientation of an RFID tag relative to a reader antenna as well as surroundings, a variety of battery-free RFID sensing systems are proposed for object localization, direction tracking, material recognition, human breathing/heartbeat rate assessment, liquid leakage detection, etc. However, some technical challenges still remain to be addressed in these purely RFID-based systems. This thesis introduces computer vision (CV) techniques into RFID systems to minimize the impact of RF phase periodicity and multipath interference. In the thesis, three categories of CV-assisted battery-free RFID systems for object recognition, localization and orientation are designed, and the main contributions include: 1) This thesis presents RF-Focus, a CV-assisted system that recognizes moving RFID-tagged objects within the region of interest and tracks their trajectories in multipath environments. To achieve RF-Focus, novel RSSI/RF phase-distance models with additional multipath terms compared to traditional models are proposed to characterize the impact of multipath interference, and thereby a dual-reader-antenna solution is designed to deal with it. Moreover, the multipath terms in RSSI and RF phase can be leveraged to clean the phase shift caused by frequency-dependent RFID hardware characteristics in RF phase. After that, an innovative fusion algorithm is designed to match position proposals outputted by a 2D camera and the cleaned RF phase for object recognition. In the experiments, RF-Focus achieves 91.67% ROI object recognition in multipath environments when simultaneously tracking five moving objects. 2) This thesis proposes RF-MVO, a CV-assisted system that locates stationary RFID tags in 3D space without driving a platform carrying reader antennas along a predefined trajectory or pre-deployed track. To achieve RF-MVO, a 2D camera is affixed to reader antennas. A fusion model is designed to fuse camera trajectory in the camera view with depth-enabled RF phase to achieve real-world trajectory transformation and tag DOA estimation. On this basis, a novel 3D localization is proposed, which could avoid consuming huge computations to search for all possible regions. In addition, a joint optimization algorithm is designed to accelerate RFMVO and improve its estimation accuracy. Finally, this thesis introduces horizontal dilution of precision widely used in satellite positioning systems to find out the optimal localization result. The experiments show that RF-MVO achieves 6.23cm localization accuracy in 3D space. 3) This thesis proposes RF-Orien3D, a CV-assisted system that leverages the variation of each tag radiation pattern in a two-RFID-tag array to estimate a labelled object’s spatial directions (i.e., azimuth and elevation) in multipath environments. To achieve RF-Orien3D, this work proposes novel RSSI/RF phase-distance models when tag mutual coupling and multipath interference both occur. In the models, one variable to be estimated is tag radiation pattern, which is simulated by building a two-tag array from a 2D image; another is modulation factor, which is estimated using RFID fingerprints in non-coupling and coupling in free space. On this basis, a convolutional neural network (CNN)-based method is proposed by simulating all multipath impacts on RFID fingerprints based on the proposed fingerprint models to pre-train a CNN and then collecting measured data to fine-tune the CNN for 3D orientation. In the experiments, RF-Orien3D achieves median angle errors of 29° and 11° in azimuth and elevation